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(54) Method and apparatus for producing a waveform using a packet stream 



(57) Packet stream is generated by combining a plu- 
rality of packets corresponding to style-of-rendition 
identification information which are selected from 
among a number of packets usable for producing wave- 
forms corresponding to various styles of rendition. 
Then, a waveform having characteristics of the style of 
rendition indicated by the style-of-rendition identification 
information is produced on the basis of the generated 
packet stream. The packet stream includes a plurality of 



packets and time information of the individual packets 
and controls the pitch, amplitude and shape of the 
waveform to be produced. By thus combining packets 
corresponding to the style-of-rendition identification 
information and producing a waveform on the basis of 
the packet stream, there can be provided a waveform 
corresponding to a desired style of rendition in a simpli- 
fied manner with great facility. 
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Description 

[0001] The present invention relates generally to 
apparatus and methods for producing waveforms of 
musical tones, voices or other sounds on the basis of 
waveform data supplied from memory or the like, and 
more particularly to an improved waveform producing 
apparatus and method capable of producing waveforms 
that faithfully represent tone color variations effected by 
a player using various styles of rendition (i.e., perform- 
ing techniques) or various sorts of articulation unique to 
a natural musical instrument. It should be appreciated 
that the basic principles of the present invention can be 
applied extensively to every type of equipment, appara- 
tus and methods having the function of generating 
musical tones, voices or any other sounds, such as 
automatic performance devices, computers, electronic 
game devices and multimedia-related devices, not to 
mention electronic musical instruments. Also, let it be 
assumed that the terms tone waveform" in this specifi- 
cation are not necessarily limited to a waveform of a 
musical tone alone and are used in a much broader 
sense that may embrace a waveform of a voice or arty 
other type of sound. 

[0002] The so-called ■waveform memory readout" 
technique has already been well known, which 
p restores waveform data (i.e., waveform sample data) 
coded in a given coding scheme, such as the PCM 
(Pulse Code Modulation), DPCM (Differential Pulse 
Code Modulation) or ADPCM (Adaptive Differential 
Pulse Code Modulation), and then reads out the thus- 
prestored waveform data at a rate corresponding to a 
desired tone pitch to thereby produce a tone waveform. 
So far, various types of ■waveform memory readout" 
technique have been proposed and known in the art, 
most of which are directed to producing a waveform 
covering from the start to end of a tone. As one specific 
example of the waveform memory readout technique, 
there has been known a scheme of prestoring waveform 
data of a complete waveform of a tone covering from the 
start to end thereof. As another example of the wave- 
form memory readout technique, there has been known 
a scheme of prestoring waveform data of a complete 
waveform only for a particular portion, such as an attach 
portion, of a tone presenting relatively complex varia- 
tions and prestoring a predetermined loop waveform for 
a sustain portion and the like presenting less variations. 
In this patent specification, the terms "loop waveform" 
are used to refer to a waveform to be read out repeat- 
edly, i.e., in a looped fashion. 

[0003] With the conventional waveform memory 
readout scheme of prestoring waveform data of a com- 
plete waveform of a tone covering from the start to end 
thereof or prestoring waveform data of a complete 
waveform only for a particular portion, such as an attach 
portion, of a tone, however, it has been necessary to 
prestore a great number of various waveform data cor- 
responding to a variety of styles of rendition (or various 



sorts of articulation), which would undesirably require a 
large storage capacity. 

[0004] Further, although the above-mentioned 
scheme of prestoring waveform data of a complete 

5 waveform of a tone can faithfully express tone color var- 
iations effected using various styles of rendition (or var- 
ious sorts of articulation) unique to a natural musical 
instrument, it can only reproduce the tone in just the 
same way as the prestored waveform data and thus 

10 would afford very poor controllability and editabifity. For 
instance, with this waveform memory readout scheme, 
it has been extremely difficult to control time-axial and 
other characteristics of the waveform data, correspond- 
ing to a desired style of rendition (or sort of articulation), 

is in accordance with performance data. 

[0005] It is therefore an object of the present inven- 
tion to provide a waveform producing method and appa- 
ratus which can produce high-quality waveform data 
corresponding to a variety of styles of rendition (or vari- 

20 ous sorts of articulation) in a simplified manner with 
greatly increased facility and controllability. 
[0006] In order to accomplish the above-mentioned 
object, the present invention provides a waveform pro- 
ducing method which comprises the steps of: receiving 

25 style-of -rendition identification information representing 0 
a style of rendition of a performance tone; generating a 
packet stream in accordance with the received style-of- 
rendition identification information, the packet stream 
including packets of information for producing and/or 

30 controlling a waveform; and producing a waveform on 
the basis of the generated packet stream. 
[0007] Packet stream is generated by combining a 
plurality of packets corresponding to the received style- 
of-rendition identification information which are selected 

35 from among a number of packets usable for producing 
waveforms corresponding to various styles of rendition. 
Then, a waveform having characteristics of the style of 
rendition indicated by the style-of-rendition identification 
information is produced on the basis of the generated 

40 packet stream. By thus combining the packets corre- 
sponding to the style-of-rendition identification informa- 
tion to generate a packet stream and producing a 
waveform on the basis of the generated packet stream, 
there can be provided a waveform corresponding to a 

45 desired style of rendition in a simplified manner with 
great facility. 

[0008] For example, the style-of-rendition identifica- 
tion (ID) information may be supplied, according to the 
characteristics of the style of rendition of the perform- 
so ance tone, in correspondence with a partial tone seg- 
ment such as an attack, body or release portion, or in 
correspondence with a link or joint segment between 
adjoining tones such as a slur, or in correspondence 
with a specially performed tone segment such as a 
55 vibrato, or in correspondence with a plurality of notes 
constituting a phrase. Namely, given style-of-rendition 
identification (ID) information is supplied in accordance 
with a performance to be reproduced. In the present 
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invention, once such style-of-rendition identification (ID) 
information is received, a packet stream for producing a 
waveform representing the style of rendition is gener- 
ated by combining a plurality of packets corresponding 
to the received style-of-rendition identification informa- § 
tion. 

[0009] The packet stream includes a plurality of 
packets and time information of the individual packets. 
For example, the time information pertains to any of 
time factors such as a time length of a waveform section 10 
corresponding to the packet and note-on timing and 
note-off timing in the waveform section. Waveform or 
envelope segments corresponding to various waveform 
factors can be built along a reproducing time axis of the 
performance tone by arranging the individual packets is 
on the time axis on the basis of the time information. 
Thus, a waveform of the performance tone is produced 
on the basis of the individual packets arranged on the 
time axis. There can be produced a performance tone 
waveform presenting characteristics of the style of ren- 20 
d'rtion corresponding to the style-of-rendition identifica- 
tion information, for example, by imparting, to the 
waveform to be produced, a pitch and amplitude and 
time variation characteristics of the pitch and amplitude 
corresponding to the packet stream. 25 
[0010] In the present invention, the step of produc- 
ing a waveform may include a step of adjusting the time 
information and a step of arranging the packets on a 
time axis on the basis of the adjusted time information 
adjusted. The step of adjusting the time information may 30 
adjust the time information forward or backward by a 
predetermined amount The forward or backward 
adjustment of the time information by the predetermined 
amount may be performed on the basis of a random 
number. 35 
[0011] In this way, the time information of the indi- 
vidual packets in the packet stream is adjusted, and the 
packets are arranged on the time axis on the basis of 
the adjusted time information. Namely, waveform or 
envelope segments corresponding to various factors of 40 
a waveform can be modified along the reproducing time 
axis of the performance tone. Because a waveform can 
be produced along the thus-modified reproducing time 
axis, the present invention allows style-of-rendition 
waveforms rich in variations to be produced with a sim- 45 
plified structure and increased controllability. 
[0012] The present invention also provides a wave- 
form producing method which comprises the steps of: 
receiving a packet stream including a plurality of pack- 
ets and time information of individual ones of the pack- so 
ets; arranging, on a time axis, vector data for producing 
a waveform representing a style of rendition of a per- 
formance tone corresponding to the packets, in accord- 
ance with the time information; and producing a 
waveform on the basis of the vector data arranged on 55 
the time axis. 

[0013] In this invention, the vector data for produc- 
ing a waveform representing the style of rendition of the 



4 

performance tone corresponding to the packets in the 
packet stream is arranged on the time axis in accord- 
ance with the respective time information of the individ- 
ual packets, and a waveform is produced on the basis of 
the vector data. Here, the vector data corresponds, for 
example, to various fundamental waveform factors for 
producing the waveform, such as a waveform shape 
(that determines a tone color or timbre), pitch variation 
over time and amplitude variation over time, and the 
vectors of these waveform factors will hereinafter be 
called a waveform shape vector, pitch vector and ampli- 
tude vector, respectively. Time vector representing a 
progression of the time axis of the waveform may also 
be used. The respective time axes of the waveform 
shape vector, pitch vector, amplitude vector, etc. can be 
controlled in accordance with the time vector. 
[0014] Waveform or envelope segments corre- 
sponding to various waveform factors can be built along 
the reproducing time axis of the performance tone by 
arranging the individual vector data on the time axis. 
Thus, a waveform of the performance tone is produced 
on the basis of the individual vector data on arranged 
the time axis. There can be produced a performance 
tone waveform presenting characteristics of the style of 
rendition corresponding to the packets can be pro- 
duced, for example, by imparting, to the waveform 
shape vector, a pitch and time variation characteristics 
of the pitch corresponding to the pitch vector and ampli- 
tude and time variation characteristics of the amplitude 
corresponding to the amplitude vector. 
[0015] The present invention may be constructed 
and implemented not only as the method invention as 
discussed above but also as an apparatus invention. 
Also, the present invention may be arranged and imple- 
mented as a software program for execution by a proc- 
essor such as a computer or DSP, as well as a storage 
medium storing such a program. Furthermore, the proc- 
essor used in the present invention may comprise a 
dedicated processor based on predetermined fixed 
hardware circuitry, rather than a general-purpose type 
processor capable of running software. 
[0016] For better understanding of the object and 
other features of the present invention, its preferred 
embodiments will be described in greater detail herein- 
below with reference to the accompanying drawings, in 
which: 

Fig. 1 is a block diagram showing an exemplary 
hardware organization of a waveform producing 
apparatus in accordance with a preferred embodi- 
ment of the present invention; 
Fig. 2 is a flow chart showing an exemplary opera- 
tional sequence of a waveform database creation 
process carried out in the waveform producing 
apparatus of Fig. 1 ; 

Fig. 3 is a diagram schematically illustrating various 
waveform components and factors constituting an 
actual waveform segment corresponding to a style- 
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of-rend*rtion module; 

Rg. 4A is a flow chart showing an exemplary oper- 
ational sequence of a database-based tone synthe- 
sis process; 

Fig. 4B is a block diagram showing an example of a 
dedicated hardware apparatus constructed to carry 
out a tone synthesis process similar to that of Rg. 
4A; 

Rg. 5 is a flow chart showing an exemplary opera- 
tional sequence of a style-of-rendition synthesis 
process performed by a style-of-rendition synthesis 
section shown in Rg. 4B; 

Rg. 6 is a flow chart showing in greater detail the 
style-of-rendition synthesis process performed by 
the style-of-rendition synthesis section; 
Rg. 7 is a flow chart showing an exemplary opera- 
tional sequence of a waveform linking process in 
relation to a case where the style-of-rendition mod- 
ules each corresponds to an amplitude or pitch fac- 
tor; 

Rg. 8A is a conceptual diagram explanatory of a 
waveform thinning-out operation performed when 
an attack-portion waveform and a body-portion 
waveform are interconnected; 
Rg. 8B is a conceptual diagram explanatory of a 
waveform thinning-out operation performed when a 
body-portion waveform and a release-portion wave- 
form are interconnected; 

Rg. 8C is a conceptual diagram explanatory of a 
waveform thinning-out operation performed when a 
bend-attack-portion waveform and a release-por- 
tion waveform are interconnected; 
Rg. 8D is a conceptual diagram explanatory of a 
waveform thinning-out operation performed when a 
normal-attack-portion waveform and a release-por- 
tion waveform having a loop waveform segment are 
interconnected; 

Rg. 9 is a conceptual diagram explanatory of a 
waveform linking process performed in a situation 
where a sampled length of a style-of-rendition mod- 
ule is so short that it would end before another 
style-of-rendition module following the same starts; 
Rg. 10 is a conceptual diagram explanatory of 
exemplary packet streams; 
Rg. 11 is a conceptual block diagram of a general 
organization of a waveform synthesis section, 
which is also explanatory of behavior of the wave- 
form synthesis section; 

Rg. 12 is a block diagram outlining a general oper- 
ational flow of the waveform synthesis; 
Rg. 13 is a block diagram explanatory of a vector 
loader; 

Rg. 14 is a block diagram explanatory of a vector 
operator; 

Rg. 15 is a block diagram explanatory of a vector 
decoder; and 

Rg. 1 6 is a diagram conceptually showing an exem- 
plary data organization of vector data. 



[0017] Rg. 1 is a block diagram showing an exem- 
plary hardware organization of a waveform producing 
apparatus in accordance with a preferred embodiment 
of the present invention. The waveform producing appa- 
5 ratus illustrated here is constructed using a computer, 
and predetermined waveform producing processing is 
carried out by the computer executing predetermined 
waveform producing programs (software). Of course, 
the waveform producing processing may be imple- 
to mented by microprograms for execution by a DSP (Dig- 
ital Signal Processor), rather than by such computer 
software. Also, the waveform producing processing of 
the invention may be implemented by a dedicated hard- 
ware apparatus that includes discrete circuits or inte- 
rs grated or large-scale integrated circuit Further, the 
waveform producing apparatus of the invention may be 
implemented as an electronic musical instrument, kara- 
oke device, electronic game device, multimedia-related 
device, personal computer or any other desired form of 
20 product. 

[0018] In Fig. 1, the waveform producing apparatus 
in accordance with the preferred embodiment of the 
present invention includes a CPU (Central Processing 
Unit) 101 functioning as a main control section of the 

25 computer, to which are connected, via a bus (e.g., data 
and address bus) BL, a ROM (Read-Only Memory) 102, 
a RAM (Random Access Memory) 103, a switch panel 
104, a panel display unit 105, a drive 106, a waveform 
input section 107, a waveform output section 108, a 

30 hard disk 109 and a communication interface 111. The 
CPU 101 carries out various processes directed to 
"waveform database creation", "tone synthesis based 
on the created database (software tone generator)", etc. 
on the basis of predetermined programs, as will be later 

35 described in detail. These programs are supplied, for 
example, from a network via the communication inter- 
face 111 or from an external storage medium 106A, 
such as a CD or MO (Magneto-Optical disk) mounted to 
the drive 106, and then stored in the hard disk 109. In 

40 execution of a desired one of the programs, the desired 
program is loaded from the hard disk 1 09 into the RAM 
103; however, the programs may be prestored in the 
ROM 102. 

[0019] The ROM 102 stores therein various pro- 
45 grams and data to be executed or referred to by the 
CPU 101. The RAM 103 is used as a working memory 
for temporarily storing various performance-related 
information and various data generated as the CPU 1 01 
executes the programs, or as a memory for storing a 
so currently-executed program and data related to the pro- 
gram. Predetermined address regions of the RAM 103 
are allocated to various functions and used as various 
registers, flags, tables, memories, etc. The switch panel 
1 04 includes various operators for instructing tone sam- 
55 pling, editing the sampled waveform data, entering vari- 
ous pieces of information, etc. The switch panel 104 
may be, for example, in the form of a ten-button keypad 
for inputting numerical value data, keyboard for inputting 
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character data or panel switches. The switch panel 1 04 
may also include other operators for selecting, setting 
and controlling a pitch, color, effect, etc. of each tone to 
be generated. The panel display unit 105 displays vari- 
ous information inputted by the switch panel, the sam- 
pled waveform data, etc. and comprises, for example, a 
liquid crystal display (LCD), CRT (Cathode Ray Tube) 
and/or the like. 

[0020] The waveform input section 1 07 contains an 
A/D converter for converting an analog tone signal, 
introduced via an external waveform input device such 
as a microphone, into digital data (waveform data sam- 
pling), and inputs the thus-sampled digital waveform 
data into the RAM 103 or hard disk 109 as original 
waveform data from which to produce desired waveform 
data. In the "waveform database creation" process car- 
ried out by the CPU 101, a waveform database of the 
present invention is created on the basts of the above- 
mentioned original waveform data. Also, in the "data- 
base-based tone synthesis" process earned out by the 
CPU 101, waveform data of each tone signal corre- 
sponding to performance information are produced 
using the above-mentioned waveform database. Of 
course, in the instant embodiment, a plurality of tone 
signals can be generated simultaneously. The thus-pro- 
duced waveform data of each tone signal are given via 
the bus BL to the waveform output section 1 08 and then 
stored into a buffer thereof as necessary. The waveform 
output section 108 reads out the buffered waveform 
data at a predetermined output sampling frequency and 
then sends the waveform data to a sound system 108A 
after D/A-converting the data. In this way, each tone sig- 
nal output from the waveform output section 108 is 
sounded or audibly reproduced via the sound system 
108A. Here, the hard disk 109 is provided for storing 
data (various data of a later-described style-of-rendition 
table, code book, etc.) for synthesizing a desired wave- 
form corresponding to waveform data and style of rendi- 
tion, a plurality of sorts of performance-related data 
such as tone color data composed of various tone color 
parameters, and control-related data such as those of 
various programs to be executed by the CPU 101. 
[0021] The drive 106 functions to drive a removable 
disk (external storage medium 106A) for storing data 
(various data of the later-described style-of-rendition 
table, code book, etc.) for synthesizing a desired wave- 
form corresponding to waveform data and style of rendi- 
tion, a plurality of sorts of performance-related data 
such as tone color data composed of various tone color 
parameters and control-related data such as those of 
various programs to be executed by the CPU 1 01 . The 
external storage medium 106A to be driven by the drive 
106 may be any one of various known removable-type 
media, such as a floppy disk (FD), compact disk (CD- 
ROM or CD-RAM), magneto-optical (MO) disk or digital 
versatile disk (DVD). Stored contents (control program) 
of the external storage medium 1 06A set in the drive 
106 may be loaded directly into the RAM 103, without 
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being first loaded into the hard disk 1 09. The approach 
of supplying a desired program via the external storage 
medium 106A or via a communication network is very 
advantageous in that it can greatly facilitate version 
5 upgrade of the control program, addition of a new con- 
trol program, etc. 

[0022] Further, the communication interface 1 11 is 
connected to a communication network, such as a LAN 
(Local Area Network), the Internet or telephone lines, 

10 via which it may be connected to a desired sever com- 
puter or the like (not shown) so as to input a control pro- 
gram and various data or performance information to 
the waveform producing apparatus. Namely, in a situa- 
tion where the control program and various data are not 

is contained in the ROM 1 02 or hard disk 1 09 of the wave- 
form producing apparatus, these control program and 
data can be downloaded from the server computer via 
the communication interface 111 to the apparatus. In 
such a case, the waveform producing apparatus of the 

20 invention, which is a "client", sends a command to 
request the server computer to download the control 
program and various data by way of the communication 
interface 111 and communication network. In response 
to the command from the client, the server computer 

25 delivers the requested control program and data to the 
waveform producing apparatus via the communication 
network. The waveform producing apparatus receives 
the control program and data from the server computer 
via the communication network and communication 

30 interface 111 and accumulatively stores them into the 
hard disk 109. In this way, the necessary downloading 
of the control program and various data Is completed. It 
should be obvious that the waveform producing appara- - 
tus may further includes a MIDI interface so as to 

35 receive MIDI performance information. It should also be . 
obvious that a music-performing keyboard and music 
operating equipment may be connected to the bus BL 
so that performance information can be supplied to the 
waveform producing apparatus by an actual real-time 

40 performance. Of course, the external storage medium 
containing performance information of a desired music 
piece may be used to supply the performance informa- 
tion of the desired music piece. 
[0023] Fig. 2 is a flow chart showing an exemplary 

45 operational sequence of the waveform database crea- 
tion process carried out in the above-described wave- 
form producing apparatus of the invention, which is 
directed to creating vector data on the basis of wave- 
forms of tones actually performed with various styles of 

so rendition or performing techniques (or various sorts of 
articulation) in such a manner that the created vector 
data correspond to various styles of rendition (sorts of 
articulation). 

[0024] First, at step S1, a database storage 
55 medium, such as the hard disk 1 09, is provided for stor- 
ing data of the later-described style-of-rendition table 
and code book. Then, at step S2, waveform data are 
acquired which correspond to tones performed on van- 
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ous natural musical instruments with various styles of 
rendition. Namely, at this step S2, various performance 
tones actually produced on various natural musical 
instruments are acquired via an external waveform input 
device, such as a microphone, through the waveform 
input section 107, and waveform data of these perform- 
ance tones (i.e., original waveform data) are stored into 
predetermined areas of the hard disk 109. At this time, 
the waveform data of either the entire performance or 
only part of the performance, such as a particular 
phrase, one particular tone or characteristic portions 
like attack and release portions of a particular tone, may 
be acquired and stored. At following step S3, the thus- 
acquired waveform data of each of the performance 
tones corresponding to the various performance styles 
unique to the natural musical instruments are seg- 
mented every characteristic portion, then subjected to a 
tuning operation and then given file names. Namely, the 
acquired original waveform data of each of the perform- 
ance tones are segmented into partial waveforms 
(waveform segmentation), each representing a charac- 
teristic waveform shape variation, such as an attack- 
portion waveform, body-portion waveform, release-por- 
tion waveform and joint-portion waveform, the tuning 
operation is performed to determine the respective 
pitches of the individual segmented waveform data or 
partial waveforms covering one or two or more cycles of 
the tone in question, and then unique file names are 
imparted to the segmented waveform data. Note that in 
the case where only the waveform data of part of the 
performance, such as attack and release portions, are 
acquired, the above-mentioned waveform segmentation 
can be dispensed with. 

[0025] Then, at step S4, the waveform data having 
been processed at step S3 are divided into waveform 
components through frequency analysis. Namely, each 
of the segmented partial waveforms is subjected to Fast 
Fourier Transform (FFT) for division into a plurality of 
waveform components (in the instant embodiment, har- 
monic and nonharmonic components). In addition, 
characteristics of various waveform factors, such as a 
waveform shape, pitch and amplitude, are extracted 
from each of the waveform components (harmonic and 
nonharmonic components); however, in the case where 
the each of the segmented partial waveforms is divided 
into the harmonic and nonharmonic components, the 
pitch extraction from the nonharmonic component may 
be omitted because the nonharmonic component has 
no pitch. For example, the "waveform shape" (timbre) 
factor represents extracted characteristics of a wave- 
form shape normalized in pitch and amplitude, the 
"pitch" factor represents extracted characteristics of a 
pitch variation from a predetermined reference pitch, 
and the "amplitude" factor represents extracted charac- 
teristics of an amplitude envelope. 
[0026] At next step S5, vector data are created. 
Namely, for each of the waveform shape (timbre), pitch 
and amplitude factors of the divided waveform compo- 



nents (e.g., harmonic and nonharmonic components), a 
plurality of sample values of succussive sample points 
are extracted dispersedly or, if necessary, successively, 
and each extracted sample value group of succussive 

s sample points thus obtained is given a different or 
unique vector ID (identification information) and stored 
into the code book along with data indicative of a time 
position thereof. Hereinafter, such sample data are 
referred to as "vector data". The instant embodiment 

10 creates vector data of the waveform shape (timbre) fac- 
tor, pitch factor and amplitude factor of each of the har- 
monic components, and vector data of the waveform 
shape (timbre) factor and amplitude factor of each of the 
nonharmonic components. The vector data of each of 

75 the waveform factors is data variable in accordance with 
the passage of time along the time axis. Then, as will be 
later described later, data of style-of -rendition modules 
are created to store the style-of-rendition modules into 
the style-of-rendition table. The thus-created style-of- 

20 rendition modules and vector data are written into the 
style-of-rendition table and code book in the database 
for data accumulation into the database, at step S6. As 
noted above, the vector data differ from the original 
waveform data as initially introduced into the waveform 

25 producing apparatus of the invention; these are the data 
obtained by dividing the introduced original waveform 
for each of the waveform factors. Each of the vector data 
is data that ultimately becomes a minimum constituent 
unit of a style-of-rendition module. Thus, in the code 

30 book, the extracted partial waveform data representing 
respective variations in the waveform shape are stored 
in compressed form. In the style-of-rendition table, on 
the other hand, data of various style-of-rendition mod- 
ules are stored, such as various data necessary forcon- 

35 verting the vector data, stored in compressed form, 
back to the waveform data of the original waveform 
shape and ID data for designating a desired one of the 
vector data stored in the code book, as will be later 
described in detail. 

40 [0027] During the extraction of the characteristics of 
the various waveform factors at step S4, characteristics 
of a time factor are also extracted in addition to those of 
the above-mentioned amplitude, pitch and waveform 
shape factors. Hereinafter, thus-extracted vector data of 

45 the time factor will be referred to as "time vector data". 
The time length of part of the original waveform data, 
corresponding to the time section of the extracted par- 
tial waveform data, is used directly as the time factor. 
Thus, if the original time length (variable value) of the 

so time section in question is represented by a ratio "1", 
then there is no need to analyze and measure the time 
length in this waveform database creation process. 
Because, in this case, the data of the time factor, i.e. 
time vector data, has the same value "1" in each of the 

55 time sections, the time length need not necessarily be 
stored in the code book. However, the present invention 
is, of course, not so limited and may be modified to ana- 
lyze and measure the actual time length and store the 
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thus-measured time length into the code book as the 
time vector data. 

[0028] Then, at step S7 of Fig. 2, a determination is 
made as to whether the database creation has been 
executed to a sufficient degree, i.e. whether or not a suf- 
ficient quantity of style-of-rendition module data and 
vector data have been obtained by acquiring, via the 
externa] waveform input device, a sufficient quantity of 
original waveform data of tones performed on various 
natural musical instruments with various styles of rendi- 
tion. The determination at step S7 is not necessarily lim- 
ited to an automatic determination and may be made on 
the basis of a user's manual switch input operation giv- 
ing an instruction as to whether the waveform database 
creation process should be continued or not If the 
acquisition of the original waveform data and creation of 
the vector data based thereon has been executed to a 
sufficient degree (YES determination at step S7), the 
instant waveform database creation process is brought 
to an end. If, on the other hand, the acquisition of the 
original waveform data and creation of the vector data 
based thereon has not yet been executed to a sufficient 
degree and hence has to be executed further (NO 
determination at step S7), the waveform database crea- 
tion process loops back to step S2 in order to repeat the 
above-described operations of steps S2 - S7. The 
determination of step S7 as to whether the database 
creation has been executed to a sufficient degree may 
be made by actually using the created vector data to 
generate tones on a trial basis. Namely, after the 
sequence of the waveform database creation process of 
Fig. 2 is terminated by provisionally determining at step 
S7 that a sufficient quantity of vector data have been 
created, there may be performed an operation of actu- 
ally using the created vector data to generate tones on 
a trial basis and then, if the thus-generated tones are 
found to be unsatisfactory as a result of the trial tone 
generation, repeating the operations at and after step 
S2 to create further vector data. Namely, in the instant 
embodiment, the operation of creating further vector 
data is performed on an as-needed basis. 
[0029] It should be appreciated here that the above- 
described waveform database creation process may be 
arranged to add/delete any desired styie-of-rendition 
module or edit the data of a desired style-of-rendition 
module. 

[0030] Now, the following paragraphs describe the 
style-of-rendition module data in greater detail. 
[0031] Each of the style-of-rendition modules is 
stored in the style-of-rendition table arranged as a data- 
base in the hard disk 109 and can be designated by a 
combination of "style-of-rendition ID" and ■style-of-ren- 
dition parameters". The style-of-rendition ID contains 
musical instrument type information and module part 
name and can be defined, for example, as follows. 
Assuming that each style-of-rendition ID consists of 32 
bits (Oth - 31st bits), six bits of the 32 bits are use for the 
musical instrument type information. In the instant 



12 

embodiment, for example, if the six-bit train constituting 
the musical instrument type information is "000000", it is 
indicative of "AltoSax" (an alto saxhorn), and if the six- 
bit train constituting the musical instrument type infor- 

5 mation is "001000", it is indicative of "Violin" (a violin); 
note that the upper three bits of the six-bit train may be 
used to represent a major class of the musical instru- 
ment while the lower three bits may be used to repre- 
sent a minor class of the musical instrument Further, 

to other six bits of the 32 bits are used for the module part 
name. If the six-bit train constituting the module part 
name is "000000", it is indicative of a module part name 
"NormalAttack", if the six-bit train is "000001 ", it is indic- 
ative of "BendAttack", if the six-bit train is "00001 0", it is 

75 indicative of "GraceNoteAttack", if the six-bit train is 
"001000", ft is indicative of "NormalShortBoay, if the 
six-bit train is "001001", it is indicative of "VibBody", if 
the six-bit train is "001010", it is indicative of "Normal- 
LongBody", if the six-bit train is "01 0000", it is indicative 

20 of "NormalRelease", if the six-bit train is "01 1 000", it is 
indicative of "NormaUoint" and if the six-bit train is 
"01 1 001", it is indicative of "GraceNoteJoint". Of course, 
the present invention is not limited to the above-noted 
arrangements. 

25 [0032] As stated above, each individual style-of- 
rendition module is specified by a combination of the 
"style-of-rendition ID" and "style-of-rendition parame- 
ters"; that is, a predetermined style-of-rendition module 
can be specified in accordance with the style-of-rendi- 

30 tion ID and its contents can be variably controlled in 
accordance with the style-of-rendition parameters. The 
style-of-rendition parameters are parameters for char- 
acterizing or controlling the waveform data correspond- 
ing to the style-of-rendition module, and predetermined v 

35 sorts of style-of-rendition parameters are provided for 
each style-of-rendition module. For example, for the 
"AltoSaxfNormalAttack]" module, there may be pro- 
vided style-of-rendition parameters pertaining to an 
absolute tone pitch and tone volume immediately follow- 

40 ing the attack, etc. For the "AltoSax[BendUpAttack]" 
module, there may be provided style-of-rendition 
parameters pertaining to an absolute tone pitch at the 
end of the bendup attack, initial value of a bend depth at 
the time of the bendup attack, time length from the start 

45 (note-on timing) to end of the bendup attack, tone vol- 
ume immediately following the attack, timewise 
stretch/contraction of a default curve during the bendup 
attack, etc. For the "AltoSax[NormalShortBody] B mod- 
ule, there may be provided style-of-rendition parame- 

50 ters pertaining to an absolute tone pitch of the style-of- 
rendition module, end and start times of the normal 
short body, dynamics at the start and end of the normal 
short body, etc. Note that the style-of-rendition module 
does not necessarily include data (later-described 

55 waveform factor data) corresponding to all the values 
which the style-of-rendition parameters can take; the 
style-of-rendition module may include data correspond- 
ing to only some discrete (non-successive) values of the 
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style-of-rendition parameters. That is, for the 
"AltoSax[NormatAttack] a module, for example, there 
may be stored data corresponding to only some, not all, 
of the values representative of the absolute tone pitch 
and tone volume immediately following the attack. 
[0033] By thus allowing each style-of-rendition 
module to be specified by a combination of the style-of- 
rendition ID and style-of-rendition parameters, It is pos- 
sible to designate data corresponding to a desired set of 
style-of-rendition parameters from among a plurality of 
data (waveform factor data) indicative of the normal 
attack portion of an alto saxophone tone, for example, in 
the case of the "ArtoSax[NormalAttack]" module. In the 
case of the "Violin[BendAttack]" module, it is possible to 
designate data corresponding to a desired set of style- 
of-rendition parameters from among a plurality of data 
(waveform factor data) indicative of the bend attack por- 
tion of a violin tone. 

[0034] In the style-of-rendition table, there are 
stored, for each individual style-of-rendition module, 
data necessary for producing a waveform correspond- 
ing to the style-of-rendition module, such as vector IDs 
designating the vector data of the individual waveform 
factors (e.g., the waveform shape factor, pitch factor 
(pitch envelope) and amplitude factor (amplitude enve- 
lope)), train of values at representative points (i.e., data 
indicative of representative sample points to be modi- 
fied in a train of a plurality of samples), and respective 
starting and ending time positions of the vector data of 
the individual waveform factors (e.g., the waveform 
shape factor, pitch factor (pitch envelope) and amplitude 
factor (amplitude envelope)). Namely, in the style-of- 
rendition table, there are stored various data necessary 
for reproducing a waveform of a normal shape from a 
waveform stored in the database in the form of com- 
pressed vector data; hereinafter, such data will also be 
called "waveform factor data". The following explain 
details of one of the data groups stored in the style-of- 
rendition table in association with various style-of-rendi- 
tion modules, and more particularly is explanatory of the 
data group stored for the AloSax[NormalAttack] mod- 
ule: 

Data 1: Sampled length of the style-of-rendition 
module; 

Data 2: Position of note-on timing; 
Data 3: Vector ID of the amplitude factor of the har- 
monic component and train of the representative 
point values; 

Data 4: Vector ID of the pitch factor of the harmonic 
component and train of the representative point val- 
ues; 

Data 5: Vector ID of the waveform shape (timbre) 
factor of the harmonic component; 
Data 6: Vector ID of the amplitude factor of the non- 
harmonic component and train of the representa- 
tive point values; 

Data 7: Vector ID of the waveform shape (timbre) 



factor of the nonharmonic component; 

Data 8: Start position of a waveform block of the 

waveform shape (timbre) factor of the harmonic 

component; 

5 Data 9: End position of a waveform block of the 

waveform shape (timbre) factor of the harmonic 
component (i.e., start position of a loop portion of 
the waveform shape (timbre) factor of the harmonic 
component); 

10 Data 10: Start position of a waveform block of the 
waveform shape (timbre) factor of the nonharmonic 
component; 

Data 11: End position of a waveform block of the 
waveform shape (timbre) factor of the nonharmonic 
15 component; (i.e., start position of a loop portion of 
the waveform shape (timbre) factor of the nonhar- 
monic component); and 

Data 12: End position of a loop portion of the wave- 
form shape (timbre) factor of the nonharmonic com- 
20 ponent 

[0035] Data 1 - Data 12 mentioned above will be 
described below in greater detail with reference to Fig. 
3. 

25 [0036] Fig. 3 is a diagram schematically illustrating 
various waveform components and waveform factors 
constituting an actual waveform section corresponding 
to the style-of-rendition module in question. From the 
top to bottom of Fig. 3, there are shown the amplitude 

30 factor, pitch factor and waveform shape (timbre) factor 
of the harmonic component, and the amplitude factor 
and waveform shape (timbre) factor of the nonharmonic 
component which have been detected in the waveform 
section. Note that numeral values represent the respec- 

35 tive numbers of the above-mentioned data (Data 1 - 
Data 12). 

[0037] More specifically numerical value 1 repre- 
sents the sampled length of the waveform section 
(length of the waveform section) corresponding to the 

40 style-of-rendition module, which corresponds, for exam- 
ple, to the total time length of the original waveform data 
from which the style-of-rendition module is derived. 
Numerical value 2 represents the position of the note-on 
timing, which can be variably set at any time position of 

45 the style-of-rendition module. Although, in principle, 
sounding of the performance tone based on the wave- 
form is initiated at the position of the note-on timing, the 
rise start point of the waveform component may pre- 
cede the note-on timing in the case of a particular style 

so of rendition such as a bend attack. Numerical value 3 
represents the vector ID designating the vector data of 
the amplitude factor of the harmonic component and 
train of the representative point values stored in the 
code book; in the figure, two square marks filled in with 

55 black indicate these representative points. Numerical 
value 4 represents the vector ID designating the vector 
data of the pitch factor of the harmonic component and 
train of the representative point values. Numerical value 
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6 represents the vector ID designating the vector data of 
the amplitude factor of the nonharmonic component and 
train of the representative point values. The representa- 
tive point values are data to be used for changing/con- 
trolling the vector data, made up of a train of a plurality 
of samples, designated by the vector ID, and designates 
some of the representative sample points. As the 
respective time positions (plotted on the horizontal axis 
of the figure) and levels (plotted on the vertical axis of 
the figure) of the designated representative sample 
points are changed or controlled, the other sample 
points are also changed so that the overall shape of the 
vector can be changed. For example, the representative 
point values represent discrete samples fewer than the 
total number of the samples; however, the representa- 
tive point values may be values at intermediate points 
between the samples or values at a plurality of succes- 
sive samples over a predetermined range. Alternatively, 
the representative point values may be such values 
indicative of differences between the sample values, 
multipliers to be applied to the sample values or the like, 
rather than the sample values themselves. The shape of 
each vector data, i.e. shape of the envelope waveform, 
can be changed by moving the representative points 
along the horizontal axis (time axis) and/or vertical axis 
(level axis). Numerical value 5 represents the vector ID 
designating the vector data of the waveform shape (tim- 
bre) factor of the harmonic component. 
[0038] Further, In Fig. 3, numerical value 7 repre- 
sents the vector ID designating the vector data of the 
waveform shape (timbre) factor of the nonharmonic 
component. Numerical value 8 represents the start 
position of the waveform block of the waveform shape 
(timbre) factor of the harmonic component Numerical 
value 9 represents the end position of the waveform 
block of the waveform shape (timbre) factor of the har- 
monic component (i.e., the start position of the loop por- 
tion of the waveform shape (timbre) factor of the 
harmonic component). Namely, the triangle starting at a 
point denoted by m B a represents a nonloop waveform 
segment where characteristic waveform shapes are 
stored in succession, and the following rectangle start- 
ing at a point denoted by "9" represents a loop wave- 
form segment. The nonloop waveform segment 
represents a high-quality waveform segment that is 
characteristic of the style of rendition (articulation) etc. 
while the loop waveform segment represents a unit 
waveform of a relatively monotonous tone segment hav- 
ing a single or an appropriate plurality of wave cycles. 
Numerical value 10 represents the start position of the 
waveform block of the waveform shape (timbre) factor of 
the nonharmonic component Numerical value 1 1 rep- 
resents the end position of the waveform block of the 
waveform shape (timbre) factor in the nonharmonic 
component (i.e., the start position of the loop portion of 
the waveform shape (timbre) factor of the nonharmonic 
component). Further, numerical value 1 2 represents the 
end position of the loop waveform segment of the wave- 



form shape (timbre) factor in the nonharmonic compo- 
nent. Data 3 - Data 7 are ID data indicating the vector 
data stored in the code book for the individual waveform 
factors, and Data 2 and Data 8 - Data 12 are time data 

5 for restoring the original waveform (i.e., the waveform 
before the waveform segmentation) on the basis of the 
vector data. Namely, the data of each of the style-of -ren- 
dition modules comprise the data designating the vector 
data and time data. Using such style-of-rendition mod- 

io ule data stored in the style-of-rendition table and the 
waveform producing materials (i.e., vector data), any 
desired waveform can be constructed freely. Namely, 
each of the style-of-rendition modules comprises data 
representing behavior of a waveform to be produced in 

is accordance with a style of rendition or articulation. Note 
that the style-of-rendition modules may differ from each 
other in the sort and number of the data included therein 
and may include other data than the above-mentioned. 
For example, the style-of-rendition modules may include 

20 data to be used for controlling the time axis of the wave- 
form for stretch/contraction thereof (time-axial 
stretch/compression control). 

[0039] Whereas the preceding paragraphs have 
described the case where each of the style-of-rendition 

25 modules includes all of the fundamental waveform fac- 
tors (waveform shape, pitch and amplitude factors) of 
the harmonic component and the fundamental wave- 
form factors (waveform shape and amplitude factors) of 
the nonharmonic component, the present invention is 

30 not so limited, and each or some of the style-of-rendition 
modules may, of course, include only one of the wave- 
form factors (waveform shape, pitch and amplitude) of 
the harmonic component and the waveform factors 
(waveform shape and amplitude) of the nonharmonic 

35 component For example, each or some of the style-of- 
rendition modules may include a selected one or more 
of the waveform shape, pitch and amplitude factors of 
the harmonic component and waveform shape and 
amplitude factors of the nonharmonic component In 

40 this way, the style-of-rendition modules can be used 
freely in any desired combination depending on the 
waveform factor desired, which is very preferable. 
[0040] With the above-described arrangement that 
only waveform data of partial waveforms necessary for 

45 waveform shape variations (such as partial waveforms 
of attack, body, release, joint portions), rather than all 
waveform data, of tones performed on various natural 
musical instruments with various performance styles 
are extracted and stored into the hard disk 1 09 in a form 

so compressed with the data compression scheme using a 
hierarchy of the waveform components, waveform fac- 
tors and representative points, the instant embodiment 
can effectively reduce a necessary storage capacity of 
the hard disk 1 09 for storing the waveform data 

55 [0041 ] I n the waveform producing apparatus shown 
in Fig. 1 , waveform synthesis is performed by the com- 
puter executing a predetermined software program for 
the waveform synthesis process. Fig. 4A is a flow chart 
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showing an exemplary operational sequence of the pro- 
gram for the waveform synthesis process (database- 
based tone synthesis process), tn an alternative, the 
waveform synthesis process may be executed by a ded- 
icated hardware apparatus rather than the waveform 5 
synthesis program. Rg. 4B is a block diagram showing 
an example of such a dedicated hardware apparatus for 
carrying out the waveform synthesis process. The 
waveform synthesis process will be described below 
with primary reference to the block diagram of Rg. 4B w 
where corresponding steps of Rg. 4A are noted in 
parentheses in the following description; in Rg. 4A, 
hardware components corresponding to operational 
steps are denoted in parentheses. 

[0042] Music-piece-data reproduction section 101 A is 
of Rg. 4B reproduces music piece data with style-of- 
rendition marks (step S11 of Rg. 4A). For this purpose, 
the music-piece-data reproduction section 101 A 
receives the music piece data with styie-of- rendition 
marks (performance information). Generally, on a nor- 20 
mal musical score or chart, there are put various musi- 
cal marks, such as a dynamic marking (crescendo, 
decrescendo or the like), tempo mark (allegro, 
ritardando or the like), slur mark, tenuto mark and 
accent mark, which can not be MIDI data in the absence 25 
of proper conversion. Thus, these musical marks are 
converted into style-of-rendition mark data, and MIDI 
music piece data with these style-of-rendition mark data 
are provided as the "music piece data with style-of-ren- 
dition marks'. Each of the style-of-rendition mark data 30 
includes a chart ID and chart parameters. The chart ID 
is an ID indicative of the musical mark put on the musi- 
cal score, and the chart parameters are indicative of a 
degree of the particular rendition represented by the 
musical mark that is designated by the chart ID. For 35 
example, in the case where the chart ID designates a 
"vibrato", a speed, depth, etc. of the vibrato are given as 
the chart parameters, and in the case where the chart 
ID designates a "crescendo", tones volume levels at the 
start and end of the crescendo, length of a time period 40 
over which the tone volume varies, etc. are given as the 
chart parameters. 

[0043] Further, in Rg. 4B, a musical score interpre- 
tation section (player) 1 01 B carries out a musical score 
interpretation process (step S12). Specifically, the MIDI as 
data and style-of-rendition mark data (each including 
the chart ID and chart parameters) contained in the 
music piece data are converted into style-of-rendition 
designating information including style-of-rendition IDs 
and style-of-rendition parameters, which is then sent to so 
a style-of-rendition synthesis section (articulator) 101C 
along with time information. Generally, even a same 
musical mark may be interpreted differently between 
different human players so that the performance is exe- 
cuted in a different manner (i.e., with a different style of ss 
rendition or articulation) for each of the human players. 
Further, depending on an arrangement of notes or the 
like, the performance may be executed in a different 



manner for each of the human players. So, the musical 
score interpretation section 101B is provided here as a 
result of converting expertise for interpreting the marks 
(musical marks and arrangement of notes) on the musi- 
cal score into an expert system. The following are 
among various criterion for the musical score interpreta- 
tion section 1 01 B to interpret the marks on the musical 
score. For example, a vibrato can not be applied to a 
note shorter than an eighth note. With a staccato, 
dynamics increase spontaneously. Attenuation rate of a 
note depends on a degree of a tenuto. Legato does not 
cause attenuation in a tone. Speed of a vibrato of an 
eighth note is substantially determined by a time value. 
Dynamics depend on a tone pitch. Further, various 
other interpretation criterion are employed, which, for 
example, pertain to a variation in dynamics due to a 
tone pitch rise and fall within a phrase, attenuation 
dynamics linearly proportional to a sound intensity (dec- 
ibel), a variation in note length responsive to a tenuto, 
staccato or the like, and a bendup width and curve 
responsive to a bendup mark in an attack portion. The 
musical score interpretation section 101B converts the 
musical score into sounds by interpreting the musical 
score in accordance with these interpretation criterion. 
Further, the musical score interpretation section 101B 
also carries out the musical score interpretation process 
in accordance with player designation by the user, i.e. 
user's designation of a desired human player (style of 
rendition). Specifically, the musical score interpretation 
section 101 B interprets the musical score in accord- 
ance with a given mode corresponding to the desig- 
nated player or style of rendition, i.e. in a different 
manner for each designated player or style of rendition. 
For example, various different modes of interpreting a 
musical score corresponding to a plurality of human 
players are stored in the database so that the musical 
score interpretation section 1 01 B interprets the musical 
score using a selected one of the stored musical score 
interpreting modes which corresponds to the user-des- 
ignated player. 

[0044] It should be appreciated here that the music 
piece data (performance information) may be con- 
structed to include, in advance, data indicative of inter- 
preted results of the musical score. Of course, if such 
music piece data including the data indicative of inter- 
preted results have been input to the apparatus, the 
above-described musical score interpretation process 
need not be performed. Further, the musical score inter- 
pretation process may be performed by the interpreta- 
tion section 101B in a fully automatic fashion or with 
intervention of some user's manual input operations as 
appropriate. 

[0045] By referring to the style-of-rendition table on 
the basis of the converted style-of-rendition designating 
information (style-of-rendition IDs and parameters) from 
the interpretation section 101B, the style-of-rendition 
synthesis section (articulator) 101C creates a packet 
stream (also called a vector stream) corresponding to 
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thestyle-of-rendition designating information and vector 
parameters for the packet stream corresponding to the 
style-of-rendition parameters, and supplies the thus- 
created packet stream and vector parameters to a 
waveform synthesis section 1 01 D (step S13). The data 5 
supplied as the packet stream to the waveform synthe- 
sis section 101D include time information, vector IDs, 
representative point values, etc. of the packets in the 
case of the pitch and amplitude factors, and vector IDs, 
time information, etc. in the case of the waveform shape 
(timbre) factor, as will be later described in detail. 
[0046] Then, the waveform synthesis section 1 01 D 
retrieves the vector data from the code book in accord- 
ance with the supplied packet stream, changes or mod- 
ifies the retrieved vector data in accordance with the 
vector parameters, and synthesizes a waveform on the 
basis of the thus-changed vector data (step S14). After 
that, the waveform synthesis section 1 01 D carries out a 
waveform production process for another performance 
part (step S15). Here, the "other performance part" 
means any one of a plurality of performance parts which 
is not subjected to the style-of-rendition synthesis proc- 
ess but is subjected to a normal tone waveform synthe- 
sis process. For the other performance part, the tone 
generation is performed using the conventional wave- 
form-memory-based tone generator scheme. The 
waveform production process for the other performance 
part may be performed by a dedicated hardware tone 
generator, such as an externa) tone generator unit or 
tone generator card detachably attachable to a compu- 
ter. For simplicity of description, however, it is assumed 
here that the instant embodiment performs the tone 
generation corresponding to styles of rendition or artic- 
ulation only for one performance part, although the 
style-of-rendition reproduction may of course be per- 
formed for a plurality of performance parts. 
[0047] Fig. 5 is a flow chart showing an exemplary 
operational sequence of the style-of-rendition synthesis 
process performed by the above-mentioned style-of- 
rendition synthesis section 101C of Fig. 4B. Although 
the style-of-rendition modules and code book are 
shown as separately stored in Fig. 5, they are, in fact, 
stored together in the database of the hard disk 109. 
[0046] The style-of-rendition synthesis section 
101C creates various packet streams to be supplied to 
the waveform synthesis section 101 D, on the basis of 
the style-of-rendition designating information (including 
the style-of-rendition IDs and style-of-rendition parame- 
ters) and time information given from the musical score 
interpretation section 101 B. The style-of-rendition mod- 
ules employed in the style-of-rendition synthesis section 
101C for the individual tone colors are not necessarily 
fixed; rather, the user can add any new style-of-rendition 
module to the currently-stored modules and stop using 
any of the currently-stored modules. Also, the style-of- 55 
rendition synthesis section 1 01 C performs a process for 
creating information to compensate for a difference or 
discrepancy between selected waveform factor data 



and values of the style-of-rendition parameters, as well 
as a process for smoothing a connection between 
waveform characteristics of successive style-of-rendi- 
tion modules, as will be later described in detail. 
[0049] Whereas, in principle, the data are given 
from the musical score interpretation section 101B to 
the style-of-rendition synthesis section 101C, the 
present invention is not so limited. Namely, there may 
be prepared music piece data with sty leof -rendition 
designating data already interpreted by the interpreta- 
tion section 1 01 B as noted earlier, or music piece data 
with style-of-rendition designating data having style-of- 
rendition IDs and style-of-rendition parameters 
imparted thereto as a result of musical score interpreta- 
tion by a human operator. Then, the data obtained by 
reproducing the thus-prepared music piece data may be 
supplied to the style-of-rendition synthesis section 
101C. 

[0050] Rg. 6 is a flow chart showing an exemplary 
operational sequence of the style-of-rendition synthesis 
process. 

[0051] The style-of-rendition synthesis section 
101C selects one of the style-of-rendition modules 
stored in the style-of-rendition table in accordance with 
the style-of-rendition ID and style-of-rendition parame- 
ters, at step S21 ; that is, one of the style-of-rendition 
modules is selected in accordance with the style-of-ren- 
dition ID (musical instrument type information plus mod- 
ule part name) and style-of-rendition parameters sent 
from the musical score interpretation section 101B. At 
this time, the musical score interpretation section 101 B, 
before proceeding to the interpretation of the musical 
score, checks the database to see what sorts of module 
parts are currently stored in the style-of-rendition table 
in correspondence with the tone color represented by 
the musical instrument type information and designates 
the style-of-rendition ID within the bounds of the cur- 
rently-stored module parts. In case a module part not 
currently stored in the style-of-rendition table has been 
designated, then another module part having simitar 
characteristics to the designated module part may be 
selected from the style-of-rendition table. After that, a 
plurality of waveform factor data are selected in accord- 
ance with the designated style-of-rendition ID and style- 
of-rendition parameters at step S22. Namely, a particu- 
lar style-of-rendition module is specified by referring to 
the style-of-rendition table on the basis of the desig- 
nated style-of-rendition ID and style-of-rendition param- 
eters, and a plurality of waveform factor data 
corresponding to the style-of-rendition parameters are 
selected from the style-of-rendition module. In the event 
that the style-of-rendition module does not include 
waveform factor data fully matching the style-of-rendi- 
tion parameters, other waveform factor data sufficiently 
close to the values of the style-of-rendition parameters 
are selected. 

[0052] Then, at step S23, time values of selected 
positions in the waveform factor data are calculated in 
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accordance with the time information; that is, the indi- 
vidual waveform factor data are arranged at their 
respective absolute time positions on the basis of the 
time information. More specifically, corresponding abso- 
lute times of the individual waveform factor data pre- 5 
senting respective relative time positions are calculated 
on the basis of the time information. This way, respec- 
tive timing of the waveform factor data is determined 
(see Fig. 3). Then, at step S24, values of the individual 
waveform factor data are adjusted in accordance with w 
the styte-of-rendition parameters; that is, differences 
between the selected waveform factor data and the val- 
ues of the style-of-rendition parameters are compen- 
sated for at this step. For example, if the tone volume 
(style-of-rendition parameter) immediately following the 15 
attack portion of the AltoSax[NormatAttack] module, 
received from the musical score interpretation section 
101B, is at a level "95" while the tone volume immedi- 
ately following the attack portion of the AttoSax[Nor- 
mal Attack] module stored in the style-of-rendition is at a 20 
level "100", then the style-of-rendition synthesis section 
101C selects the waveform factor data of the latter 
AltoSaxfNormal Attack] module whose tone volume 
level immediately following the attack portion is "100". 
However, because the tone volume level immediately 25 
following the attack portion is still "100", adjustments 
are made to the representative points of the selected 
waveform factor data so as to modify the tone volume 
level immediately following the attack portion to "95". 
This way, the values of the selected waveform factor 30 
data are adjusted to approach the values of the received 
style-of-rendition parameters. Further, at this step, there 
is made an adjustment according to a currently-set 
microtuning value for tuning of the musical instrument, 
as well as a tone volume adjustment according to tone 35 
volume variation characteristics of the musical instru- 
ment. These adjustments are performed by changing, 
sometimes greatly, the representative point values of 
the individual waveform factor data. Namely, the repre- 
sentative point values are necessary and sufficient data 40 
for the adjustments, and various adjustments are made 
by controlling the representative point values in the 
waveform factor data. 

[0053] Note that at step S23 above, the time posi- 
tions indicated by the time information may be adjusted 45 
by adjustment information such as the above-men- 
tioned style-of-rendition parameters. For example, in a 
situation where a time position based on the perform- 
ance data and a time position indicated by the time 
information do not coincide with each other, other time so 
information indicative of another time position close to 
the time position based on the performance data may 
be selected and the time position indicated by the thus- 
selected time information may be adjusted in accord- 
ance with the performance data so that the time position 55 
information intended by the performance data can be 
obtained. Further, in a situation where the performance 
data includes variable control factors such as a touch 



and velocity, time position information based on the per- 
formance data can be variably controlled by changing 
the time position information in accordance with the var- 
iable control factors. The above-mentioned adjustment 
information include information for effecting such a time 
position adjustment. 

[0054] Further, at next step S25, a waveform linking 
process is performed for smoothing respective connect- 
ing portions of adjoining style-of-rendition modules by 
adjusting the individual waveform factor data Namely, 
the representative points of the respective connecting 
portions of the adjoining style-of-rendition modules are 
brought closer and linked with each other, so as to 
smooth the waveform characteristics of the adjoining 
style-of-rendition modules. Such a connection or wave- 
form linking process is earned out for each of the wave- 
form factors, such as the waveform shape (timbre), 
amplitude and pitch of the harmonic component, or for 
each of the waveform factors, such as the waveform 
(Timbre) and amplitude of the n on harmonic component. 
[0055] At that time, adjustments are made over a 
range from a link starting point of the preceding style-of- 
rendition module to a linking end point of the succeed- 
ing style-of-rendition module. More specif ically, the rep- 
resentative points within the range from the link starting 
point to the linking end point are adjusted on the basis 
of a "mutual approaching rate". Here, the "mutual 
approaching rate" is a parameter for performing control 
to determine a point displaced from each of the preced- 
ing and succeeding style-of-rendition modules toward 
the other where the adjoining style-of-rendition modules 
are to be interlinked, and this parameter is set in accord- 
ance with a combination of the adjoining style-of-rendi- 
tion modules. In case the adjoining style-of-rendition 
modules have not been interlinked successfully, the 
connection is smoothed by thinning out the vector IDs of 
the waveform characteristics of one of the adjoining 
style-of-rendition modules. For the thinning-out of the 
vector IDs, there are provided, in the instant embodi- 
ment, a "style-of-rendition module combination table", 
thinning-out parameter range table" to be referred to 
from the style-of-rendition module combination table, 
and a thinning-out time table" to be referred to from the 
thinning-out parameter range table. 
[0056] The waveform characteristics can also be 
interlinked smoothly through a waveform linking proc- 
ess performed by the musical score interpretation sec- 
tion 101B as follows, in place of or in addition to the 
above-described waveform linking process performed 
by the style-of-rendition synthesis section 101C. For 
example, discrete regions of the style-of-rendition 
parameters (values of the dynamics, pitch parameter, 
etc.) are linked together smoothly without regard to the 
style-of-rendition modules. In shifting from a vibrato to a 
release portion, for example, the waveform characteris- 
tics may be linked smoothly by decreasing the vibrato 
effect earlier. 

[0057] Now, the above-described waveform linking 
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process, i.e. adjustments of the individual waveform fac- 
tor data for smoothing respective connecting portions of 
adjoining style-of-rendition modules (see step S25), will 
be described in more details. First, with reference to Rg. 
7, a description is made about the waveform linking 5 
process in relation to a case where the style-of-rendition 
modules each corresponds to the amplitude or pitch 
factor. 

[0058] When there is produced a great value differ- 
ence at a waveform-interconnecting point between the io 
adjoining style-of-rendition modules due to discreteness 
between the representative point values in the respec- 
tive connecting portions of the two style-of-rendition 
modules, a "mutual approaching rate" is first deter- 
mined as an index indicating to which one of the values is 
of the preceding and succeeding style-of-rendition mod- 
ules the target value of the dynamics connecting point 
or pitch connecting point should be brought closer. Let it 
be assumed here that in the instant embodiment, such 
a mutual approaching rate is given by a table as illus- 20 
trated in Rg. 7. For example, if the vector 10 of the pre- 
ceding style-of-rendition module is "3" and the vector ID 
of the succeeding style-of-rendition module is "7", then 
a mutual approaching rate of "30" is determined via the 
table. Then, the envelope shapes of the style-of-rendi- 25 
tion modules are modified progressively from the link 
starting point of the preceding style-of-rendition module 
up to the linking end point of the succeeding style-of- 
rendition module, so as to approach the respective tar- 
get values. Also, the envelope shapes of the style-of- 30 
rendition modules are modified progressively in the 
reverse direction, i.e. from the linking end point of the 
succeeding style-of-rendition module to the link starting 
point of the preceding style-of-rendition module. More 
specifically, if the mutual approaching rate has been set 35 
as "30", then the target value for the preceding style-of- 
rendition module is "30" so that the preceding style-of- 
rendition module is adjusted to be closer to the suc- 
ceeding style-of-rendition module by 30%; in the instant 
embodiment, the last one of the representative points in 40 
the preceding style-of-rendition module is brought 
downward by 30%. At the same time, the succeeding 
style-of-rendition module is adjusted to be closer to the 
preceding style-of-rendition module by 70 (i.e., 100 - 
30)%; in the instant embodiment, the leading one of the 45 
representative points in the succeeding style-of-rendi- 
tion module is brought upward by 70%. Also, in accord- 
ance with the above-mentioned adjustments of the 
leading and last representative points, a plurality of 
other representative points of the adjoining style-of-ren- so 
d'rtion modules intervening between the link starting and 
ending points are adjusted upward and downward to 
approach the respective target values. As set out above, 
the mutual approaching is effected at a plurality of rep- 
resentative points of the preceding and succeeding 55 
style-of-rendition modules. Note that although the 
above-mentioned link starting and ending points may be 
set as desired, it is desirable to set these link starting 



and ending points to coincide exactly with desired ones 
of the representative points in that undesirable bends of 
the envelope shape occurring at the link starting and 
ending points as illustrated in the figure can be avoided. 
It should also be obvious that even where the link start- 
ing and ending points are not set to coincide with the 
desired representative points, the mutual approaching 
may be performed in such a manner as to avoid the 
undesirable bends of the envelope shape. 
[0059] It should also be appreciated that the mutual 
approaching rate may be determined in any other man- 
ner than the above-mentioned. For example, the mutual 
approaching rate may be determined on the basis of the 
style-of-rendition parameters designated before and 
after the waveform-interconnecting point, or perform- 
ance data before being converted into the style-of-ren- 
dition ID and parameters, or a combination of these 
data. Further, whereas the instant embodiment has 
been described above in relation to the case where only 
one representative point is adjusted in accordance with 
the mutual approaching rate and other representative 
points are adjusted by appropriate amounts in response 
to the adjustment of the one representative point, the 
embodiment may be modified such that a separate 
mutual approaching rate is determined for each of the 
plurality of representative points so that each of the rep- 
resentative points is adjusted by an amount as specified 
by the separate approaching rate. 
[0060] Next, a description is made about the wave- 
form linking process in relation to a case where the 
style-of-rendition modules each corresponds to the 
waveform (timbre) factor, with reference to Figs. 8A - 
8D. Specifically, Rg. 8A is a conceptual diagram explan- 
atory of a waveform thinning-out operation performed 
when an attack-portion waveform and a body-portion 
waveform are interconnected, and Rg. 8B is a concep- 
tual diagram explanatory of a waveform thinning-out 
operation performed when a body-portion waveform 
and a release-portion waveform are interconnected. In 
the illustrated example of Rg. 8A, the body-portion 
waveform consists of five loop waveform segments L1 - 
L5, each of which is reproduced in a repeated or looped 
fashion. Similarly, in the illustrated example of Fig. 8B, 
the body-portion waveform consists of six loop wave- 
form segments LV - L6'. 

[0061 ] There are a variety of schemes to adjust the 
waveform factor data (namely, schemes to perform the 
waveform linking process). As one example, the 
assignee of the present patent application proposes a 
scheme which permits a smooth connection, for exam- 
ple, between a style-of-rendition module of an attack or 
joint portion and a style-of-rendition module of a body 
portion (or between a style-of-rendition module of a 
body portion and a style-of-rendition module of a 
release or joint portion), by partially thinning out the 
waveforms. It is well known to use cross-fade synthesis 
in interconnecting waveforms. However, where there is 
only a short time t between the waveform-interconnect- 



13 



25 



EP 1 087 369 A1 



26 



ing point and the start point of the first loop waveform 
segment L1 as in the illustrated example of Fig. 8A, 
there arises a need to perform rapid cross-fade synthe- 
sis within such short time period t. if such rapid cross- 
fade waveform synthesis is performed within the very 5 
short time period between the adjoining waveforms to 
be interconnected, there would be produced a wave- 
form with undesirable great noise. Thus, the instant 
embodiment of the invention is arranged to thin out 
(delete) part of the waveforms to thereby widen the time 
interval between the two waveforms to be intercon- 
nected. Because the waveforms of the attack, release 
and joint portions are each a single integral block inca- 
pable of being thinned out, the instant embodiment thins 
out a selected one of the loop waveform segments of 
the body portion; the leading loop waveform segment 
L1 is thinned out in the example of Fig. 8A and the last 
loop waveform segment L6' is thinned out in the exam- 
ple of Fig. 8B, as denoted by rectangular marks filled in 
with black. For example, in the example of Fig. 8A, 
cross-fade synthesis is performed between the second 
loop waveform segment L2 having a relatively long time 
interval from the waveform-interconnecting point and 
the trailing waveform segment of the attack portion, and 
the leading loop waveform segment L1 is not used for 
the cross-fade synthesis. Similarly, in the example of 
Fig. 8B, cross-fade synthesis is performed between the 
fifth loop waveform segment L5* having a relatively long 
time interval from the waveform-interconnecting point 
and the release-portion waveform, and the sixth loop 
waveform segment L6' is not used for the cross-fade 
synthesis. 

[0082] Note that the joint portion as referred to 
herein is a waveform section for interconnecting adjoin- 
ing tones (or tone segments) through a desired style of 
rendition. 

[0083] Further, the instant embodiment permits a 
smooth connection between a style-of-rendition module 
of an attack portion and a style-of-rendition module of a 
release or joint portion. Figs. 8C and 8D are conceptual 
diagrams explanatory of a waveform thinning-out oper- 
ation performed when the attack-portion waveform and 
release-portion waveform are interconnected. 
[0084] In this case, waveform thinning-out of the 
style-of-rendition module of the attack portion, release 
portion or the like is sometimes possible but sometimes 
impossible. Examples of the attack portion whose style- 
of-rendition module can be subjected to the waveform 
thinning-out operation include a bendup attack portion 
that has several loop waveform segments in its latter 
half. Release-portion having several loop waveform 
segments in its former half can also be subjected to the 
waveform thinning-out operation. Thus, the instant 
embodiment thins out only the waveform of such a style- 
of-rendition module that can be subjected to the wave- 
form thinning-out operation. For example, when the 
bend attack portion and release portion are intercon- 
nected, one or more of the loop waveform segments of 



the bend attack portion are thinned out (in the illustrated 
example of Rg. 8C, only one of the loop waveform seg- 
ments is thinned out as denoted by a rectangular mark 
filled in with black). When the normal attack portion and 
release portion having loop waveform segments are 
interconnected, one or more of the loop waveform seg- 
ments of the release portion are thinned out (in the illus- 
trated example of Rg. 8D, only one of the loop waveform 
segments is thinned out as denoted by a rectangular 
mark filled in with black). 

[0065] It should be appreciated here that the loop 
waveform segment to be thinned out in the instant 
embodiment need not necessarily be the one closest to 
the waveform-interconnecting point (such as the leading 
or last loop waveform segment) and such a loop wave- 
form segment to be thinned out may be designated from 
among a plurality of loop waveform segments in accord- 
ance with predetermined priority order. 
[0066] As described above, the instant embodiment 
is constructed to perform the waveform thinning-out 
operation when adjoining style-of-rendition modules 
can not be properly interconnected within the bounds of 
certain style-of-rendition parameters. For this purpose, 
there are provided, in the instant embodiment, a 'style- 
of-rendition module combination table - , "thinning-out 
parameter range table* to be referred to from the style- 
of-rendition module combination table, and a "thinning- 
out time table" to be further referred to from the thin- 
ning-out parameter range table. The style-of-rendition 
module combination table is a table to be used for deter- 
mining predetermined parameters in accordance with a 
combination of adjoining style-of-rendition modules to 
be interconnected. The thinning-out parameter range 
table is a table to be used for determining a time range 
within which the waveform thinning-out operation is to 
be effected for each of the parameters. Further, the thin- 
ning-out time table is a table to be used for determining 
a time length of the waveform thinning-out. If a time dif- 
ference between the waveform-interconnecting point 
and the leading or last loop waveform segment L1 (or 
L6") (i.e., the time t shown in Figs. 8A - 8D) is shorter 
than a predetermined reference thinning-out time 
length, then the leading or last loop waveform segment 
is thinned out in the instant embodiment. 
[0067] Further, the following paragraphs describe 
the waveform linking process performed in a situation 
where the sampled length of a style-of-rendition module 
is so short that it would end before another style-of-ren- 
dition module following the same starts, with reference 
to Rg. 9. Here, the description is made in relation to a 
waveform shape (timbre) factor packet stream that is 
made, in the left-to-right direction (in a time-serial fash- 
ion), of four style-of-rendition modules: A.Sax[Bend- 
upAttack]; A.Sax[NormalShortBody]; 
A.Sax[v1bratoBody]; and A.Sax[NormalRelease]. Sam- 
pled lengths of the individual ones of the four style-of- 
rendition modules (waveform section lengths) are each 
denoted by "length" in the figure, "note-on" and "note- 
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off" on the top row of Fig. 9 each represent event timing 
of MIDI data, "A.Sax[BendupAttack]" etc. on the middle 
row each represent generation timing of a correspond- 
ing style of rendition ID and •note", "dynamics", "depth", 
etc. on the middle row each represent generation timing 5 
of corresponding style-of-rendition parameters. 
[0068] The A.Sax[BendupAttack] module is caused 
to start at time point to. Time point t1 represents note-on 
timing within the style-of-rendition module and is made 
to coincide with instructed note-on timing. The contents 10 
of the module in the packet stream are controlled on the 
basis of the style-of-rendition parameters such as those 
of the note, dynamics and depth. The A.Sax[Nor- 
malShortBody] module is caused to start at time point 
t2. Time point t3 represents timing when a vibrato rendi- is 
tion starts at a halfway point in the waveform-intercon- 
necting region, and this timing is determined, for 
example, on the basis of start timing of a vibrato mark 
imparted to the music piece data. Time point t5 repre- 
sents note-off timing in the A. Sax[Normal Release] mod- 20 
ule and is made to coincide with instructed note-off 
timing. Starting time point t4 of the A.SaxfNormal Re- 
lease] module is determined in accordance with the 
note-off timing of the A. SaxfNormal Release] module. 
Namely, because the note-on timing occurs at time 25 
point t1 and the corresponding note-off timing occurs at 
time point t5, actual generation of a tone in accordance 
with a waveform produced from the packet stream takes 
place over a time period from time point t1 to time point 
t5. In the case of this packet stream, the time length 30 
from time point t2 to time point t4 and the total of the 
respective sampled lengths of the A.Sax[NormalRe- 
lease] and A.Sax[VibratoBody] modules intervening 
between time point t2 and time point t4 often do not 
match each other, which must be property dealt with. 35 
For this purpose, in the instant embodiment, the total of 
the respective sampled lengths of the A.Sax[NormalRe- 
lease] and A.Sax[VibratoBody] modules is made to 
coincide with the time length from time point t2 to time 
point t4 by repeating one of the modules, changing the 40 
sampled length of the module(s) or using an appropri- 
ate combination of parts of the two modules. Namely, 
the instant embodiment of the present invention is 
arranged to perform the waveform linking process with 
appropriate adjustments between the modules as nec- 45 
essary. Specifically, in the illustrated example, the wave- 
form interlinking operation is performed between the 
A.Sax[NormalShortBody] and A.Sax[VibratoBody] 
modules with the preceding A.Sax[NormalShortBody] 
module repeated, and similarly, the waveform interlink- so 
ing operation is performed between the A.SaxfVibrato- 
Body] and A.Sax[NormalRelease] modules with the 
preceding A.Sax[VibratoBody] module repeated. 
[0069] In the case where the waveform interlinking 
operation is performed between adjoining style-of-ren- 55 
dition modules by repeating one of the modules as 
described above, the time length of the repeated mod- 
ule is variably controlled. The variable control of the 



module time length, in the illustrated example, is 
effected by moving the representative points of the 
A.Sax[NormalShortBody] or A.SaxfVibratoBody] mod- 
ule; that is, the module time length is controlled in an 
appropriate manner, such as by changing a time length 
of cross-fade connection between a plurality of loop 
waveform segments constituting the module. In the 
case of the loop waveform segment, the time length of 
the entire loop reproduction can be variably controlled 
relatively easily by varying the number of loops or loop- 
lasting time. In the case of the nonloop waveform seg- 
ment, however, its length along the time axis can not be 
variably controlled so easily. Thus, a scheme of variably 
controlling the sounding time length of the entire wave- 
form of a tone comprising nonloop and loop waveform 
segments is very preferable in that it greatly facilitates 
time stretch/compression control. For this purpose, it 
will be advantageous to employ the time stretch/com- 
pression control" (abbreviated TSC") proposed earlier 
by the assignee of the present patent application in Jap- 
anese Patent Laid-open Publication No. HEI-10- 
307586; the proposed stretch/compression control can 
be advantageously applied to variably control the time- 
axial length of a nonloop waveform corresponding to a 
particular style of rendition. 

[0070] Fig. 10 is a diagram conceptually showing 
exemplary packet streams created in the above- 
described manner. Sequentially in the top-to-bottom 
direction of Rg. 10, there are shown packet streams of 
amplitude, waveform shape (Timbre) and pitch factors 
of a harmonic component and amplitude and waveform 
shape (timbre) factors of a non harmonic component. 
Further, in Rg. 10, square marks filled in with black rep- 
resent the representative points in the amplitude, wave- 
form shape (timbre) and pitch factors of the harmonic 
component and amplitude and waveform shape (timbre) 
factors of the nonharmonic component Curves con- 
necting these representative points each represent a 
shape of a vector designated by a vector ID included in 
one of the packets in the packet stream. Further, in the 
waveform shape (timbre) factor of each of the harmonic 
and nonharmonic components, blank rectangular 
blocks L each represent a loop waveform segment and 
other rectangular blocks NL each represent a nonloop 
waveform segment Of the nonloop waveform seg- 
ments, those denoted by hatched rectangular blocks 
are particu larty characteristic nonloop waveform seg- 
ments. Further, in the illustrated example of Rg. 1 0, the 
waveform shape (timbre) factor of each of the harmonic 
and nonharmonic components comprises two vectors, 
and each of the amplitude and pitch factors of the har- 
monic component and amplitude factor of the nonhar- 
monic component comprises a single vector. 
Furthermore, for each of the harmonic and nonhar- 
monic components in the illustrated example of Rg. 10, 
the amplitude and pitch factors have no vector in their 
regions that correspond in position to the nonloop wave- 
form segment of the waveform shape (timbre) factor. 



is 



20 



25 



15 



29 



EP 1 087 369 A1 



30 



However, even in the regions corresponding in position 
to the n on loop waveform segment of the waveform 
shape (timbre) factor, each of the amplitude and pitch 
factors may have a vector so that the waveform to be 
produced is controlled in accordance with the vector. In 
the VibratoBody module, the waveform shape (timbre) 
factor of the harmonic component comprises five vec- 
tors, and each of the amplitude and pitch factors of the 
harmonic component and waveform shape (timbre) and 
amplitude factors of the nonharmonic component com- 
prises a single vector. Here, note that although the 
VibratoBody module is shown as repeated three times, 
the vector shape differs for each occurrence of the mod- 
ule; this is because different style-of-rendition parame- 
ters are designated for each occurrence of the module. 
In the instant embodiment, different waveform factor 
data are selected or different level control or time-axial 
control is performed, in accordance with the different 
style-of-rendition parameters. Further, in the Normal- 
Joint module, the waveform shape (timbre) factors of 
the harmonic component and nonharmonic component 
each comprise three vectors, and each of the amplitude 
and pitch factors of the harmonic component and ampli- 
tude factor of the nonharmonic component comprises 
two vectors. Description of the NormalBody module is 
omitted here. 

[0071] In the above-mentioned manner, the style- 
of-rendition synthesis section 101C creates a packet 
stream for each of the waveform components (i.e., har- 
monic and nonharmonic components). Each of these 
packet streams comprises a plurality of packets each 
including a vector ID and time information of the packet 
In addition, each of the amplitude and pitch factors of 
the harmonic component and amplitude factor of the 
nonharmonic component includes definite values of the 
individual representative values. Of course, the present 
invention is not so limited, and each of the packets may 
include any other information in addition to the vector ID 
and time information of the packet Thus, a packet 
stream is constructed, for each of the waveform factors, 
in accordance with the contents of the individual pack- 
ets. 

[0072] It should be appreciated that the number of 
the packet streams may differ depending on the type of 
the musical instrument or the like. 
[0073] The waveform synthesis section 1 01 D syn- 
thesizes a waveform on the basis of the packet streams 
(i.e., streams of packets each including a vector ID, time 
information, adjustment information, etc.) for each of the 
waveform factors which are supplied from the style-of- 
rendition synthesis section 1 01 C. Fig. 11 is a concep- 
tual block diagram of a general organization of the 
waveform synthesis section 101D, which is explanatory 
of behavior of the synthesis section 1 01 D. Figs. 12-15 
are block diagrams showing details of individual opera- 
tions performed by the waveform synthesis section 
101D, of which Rg. 12 is a block diagram outlining a 
general operational flow of the waveform synthesis, Fig. 



13 is a block diagram explanatory of a vector loader, 
Rg. 1 4 is a block diagram explanatory of a vector oper- 
ator and Rg. 1 5 is a block diagram explanatory of a vec- 
tor decoder. 

5 [0074] Packet streams, created for the individual 
waveform factors of the harmonic and nonharmonic 
components by the style-of-rendition synthesis section 
(articulator) 101C, are sequentially input, on a packet- 
by-packet basis, to predetermined packet queue buffers 

10 21 - 25 that are provided in the waveform synthesis sec- 
tion 1 01 D in corresponding relation to the waveform fac- 
tors of the harmonic and nonharmonic components. 
After being accumulated in the respective packet queue 
buffers 21 - 25, the packets are sent to the vector loader 

75 20 in predetermined order, and the vector loader 20 
refers to the vector ID of each of the packets to read out, 
from the code book 26, the original vector data corre- 
sponding to the vector ID (original vector data loading). 
The read-out vector data are then delivered to the vec- 

20 tor decoders 31-35 provided in corresponding relation 
to the waveform factors of the harmonic and nonhar- 
monic components, via which waveforms for the individ- 
ual waveform factors are produced in predetermined 
synchronized relation to each other. The thus-produced 

25 waveforms for the individual waveform factors are then 
passed to a mixer 38. In addition to inputting the packets 
to the packet queue buffers 21 - 25, the style-of-rendi- 
tion synthesis section (articulator) 101C performs vari- 
ous control for the waveform synthesis section 101 D, 

30 such as packet stream management (i.e., management 
pertaining to production or deletion of the individual vec- 
tor data or interconnection between the vector data) and 
reproduction control (i.e., control pertaining to produc- 
tion of a desired waveform or reproduction/stop of the 

35 produced desired waveform). 

[0075] As noted above, the packets constituting the 
packet streams, having been accumulated in the packet 
queue buffer 21, are sequentially sent to the vector 
loader 20, and the vector loader 20 reads out, from the 

40 code book 26, the original vector data corresponding to 
the vector ID of each of the packets and delivers the 
read-out vector data to the vector decoder 21 (see Rg. 
12). Some of the read-out vector data may include 
adjustment information (e.g., adjustment information 

45 pertaining to the representative points). In such a case, 
the vector loader 20 modifies the read-out original vec- 
tor data in accordance with the adjustment information 
and then outputs the packets having the modified vector 
data (which will hereinafter be called "vector information 

so data" to differentiate from the "original vector data") to 
the vector decoders 31 - 35. Namely, the vector loader 
20 reads out, from the code book 26, the original vector 
data on the basis of the vector IDs of the packets input 
from the style-of-rendition synthesis section (articulator) 

55 101C, modifies the vector data in accordance with the 
adjustment information as necessary, and then passes 
the vector packets to the respective vector decoders 31 
- 35 (see Rg. 13). Examples of the adjustment informa- 
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tion pertaining to the representative points of the above- 
mentioned vector data include various information, such 
as one for changing the time information, for example, 
on the basis of a random number. 

[0076] Further, as shown in Fig. 1 4, each of the vec- s 
tor decoders 31 - 35 generates or cancels a vector oper- 
ator for processing the input vector packet and performs 
various management as to operation of the vector oper- 
ator, such as connection/synchronization between the 
vector operators, time management and conversion into w 
parameters in vector operators input from another vec- 
tor ID stream. The vector operators 36 and 37 read out 
the vector information data and perform control of read- 
out positions (speed inputs) and gains (gain inputs) of 
the vector information data. Various parameters set in is 
the vector operators 36 and 37 are managed by the cor- 
responding vector decoder 31 - 35. The vector decoder 
31-35, which are provided in corresponding relation to 
the waveform factors, each read out the vector informa- 
tion data and time-serially produce a desired waveform. 20 
As illustratively shown in Fig. 1 5, the vector decoder 31 
produces an envelope waveform of the amplitude factor 
of the harmonic component, the vector decoder 32 pro- 
duces an envelope waveform of the pitch factor of the 
harmonic component, and the vector decoder 33 pro- 25 
duces a waveform of the waveform shape (timbre) factor 
of the harmonic component. Further, the vector decoder 
34 produces an envelope waveform of the amplitude 
factor of the nonharmonic component, and the vector 
decoder 35 produces an envelope waveform of the 30 
waveform shape (timbre) factor of the nonharmonic 
component. The vector decoder 33 produces a wave- 
form of the harmonic component which has imparted 
thereto the envelope waveform of the amplitude factor 
of the harmonic component produced by the vector 35 
decoder 31 and the envelope waveform of the pitch fac- 
tor of the harmonic component produced by the vector 
decoder 32, and then outputs the thus-produced wave- 
form to the mixer 38. Namely, for the waveform repro- 
duction, the vector decoder 33 is supplied with the 40 
above-mentioned envelope waveform of the amplitude 
factor of the harmonic component as the vector opera- 
tor for performing the gain (gain input) control, and the 
above-mentioned envelope waveform of the pitch factor 
of the harmonic component as the vector operator for 45 
performing the readout position (speed input control) 
control of the vector information data. Similarly, the vec- 
tor decoder 35 produces a waveform of the nonhar- 
monic component which has imparted thereto the 
envelope waveform of the amplitude factor of the non- so 
harmonic component produced by the vector decoder 
34 and then outputs the thus-produced waveform to the 
mixer 38. Namely, for the waveform production, the vec- 
tor decoder 35 is supplied with the above-mentioned 
envelope waveform of the amplitude factor of the non- 55 
harmonic component as a control instruction for per- 
forming the gain (gain input) control. 
[0077J Further, in the instant embodiment of the 



invention, the time-serial waveform production for the 
individual waveform factors of the harmonic and non- 
harmonic components is performed while keeping 
waveform synchronization between the vector decoders 
31 - 35. If vector packets of the waveform shape (timbre) 
and amplitude factors have been input, an amplitude 
waveform based on the vector packet of the amplitude 
factor is produced in synchronism with a waveform pro- 
ducing time based on the waveform shape (timbre) fac- 
tor vector packet The amplitude of the waveform 
produced on the basis of the waveform shape (timbre) 
factor vector packet is controlled by the thus-produced 
amplitude waveform. Further, if vector, packets of the 
waveform shape (timbre) and pitch factors have been 
input, a pitch waveform based on the pitch factor vector 
packet is synthesized in synchronism with a waveform 
producing time based on the waveform shape (timbre) 
factor vector packet, and the pitch of the waveform pro- 
duced on the basis of the waveform shape (timbre) fac- 
tor vector packet is controlled by the thus-synthesized 
pitch waveform. Further, if a vector packet of the wave- 
form shape (timbre) of the harmonic component and a 
vector packet of the waveform shape (timbre) of the 
nonharmonic component have been input, a nonhar- 
monic component based on the waveform shape (tim- 
bre) vector packet of the nonharmonic component is 
synthesized in synchronism with a harmonic component 
synthesizing time based on the waveform shape (tim- 
bre) factor vector packet of the harmonic component. 
Then, a desired tone waveform is produced by mixing 
the synthesized waveforms of the harmonic and non- 
harmonic components. 

[0078] Note that the instant embodiment may be 
arranged to permit a selection as to whether or not the 
harmonic and nonharmonic components should be syn- 
chronized. In this case, only when the synchronization 
between the harmonic and nonharmonic components 
has been selected, a nonharmonic component based 
on the waveform shape (timbre) vector packet of the 
nonharmonic component may be synthesized in syn- 
chronism with the harmonic component synthesizing 
time based on the waveform shape (timbre) factor vec- 
tor packet of the harmonic component. 
[0079] As noted previously, each of the packet 
streams consists of a plurality of packets. In the case of 
the packet stream of vector packets, for example, each 
of the vector packets include vector data; namely, the 
packet stream comprises a time series of the vector 
data. Although different in the data organization and 
meaning, the vector data of the amplitude factor, pitch 
factor and waveform shape factor appear to be funda- 
mentally the same, in principle, as viewed from the vec- 
tor operators 36 and 37. 

[0080] Finally, Fig. 16 is a diagram conceptually 
showing an exemplary data organization in the vector 
data. In the illustrated example of Fig. 16, the readout 
time position of the vector data is expressed in seconds, 
and assuming that the data readout speed is uniform, 
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each sample of the vector data corresponds to one 
sample of an output waveform. Further, in the instant 
embodiment, the minimum unit of the data readout rate 
is 1/1 200 cents (= nth power of 2); thus, if the power n is 
'O - , the data readout rate is kept uniform, if the power n 5 
is "1 .0", the data readout rate is raised by a factor of 2 
(raised by one octave in the case of the waveform shape 
(timbre) factor), or if the power n is "-1.0\ the data rea- 
dout rate is lowered by a factor of 0.5 (lowered by one 
octave in the case of the waveform shape (timbre) fac- w 
tor) (see the upper column of Fig. 16). Furthermore, in 
the code book 26, there are stored actual vector data. 
For example, the vector data of the amplitude factor or 
pitch factor comprise a series of vector point structures 
and data of representative points. The series of vector 15 
point structures comprises a sequence of sets of sam- 
ple positions and values at the individual points. For 
example, the values of the amplitude factor vector data 
are expressed in decibels, and the values of the pitch 
factor vector data are expressed in 1/1200 cents based 20 
on the assumption that MIDI note number "O" has a 
value B 0.0". Further, the data of the representative 
points are in the dword (double word) arrangement, 
where are stored index numbers of the vector point 
structures as the representative points (see the lower 25 
column of Fig. 16). Of course, the present invention 
should not be construed as limited to the above-noted 
vector data organization, and may be modified vari- 
ously. 

[0081] Note that in the case where the above- 30 
described waveform producing apparatus is applied to 
an electronic musical instrument, the electronic musical 
instrument may be of any type other than the keyboard- 
based instrument, such as a stringed, wind or percus- 
sion instrument In such a case, the present invention is 35 
of course applicable not only to such an electronic musi- 
cal instrument where all of the music-piece-data repro- 
duction section 101 A, musical score interpretation 
section 101B, style-of-rendition synthesis section 101C, 
waveform synthesis section 1 01 D and the like are incor- 40 
porated together as a unit, but also to another type of 
electronic musical instrument where the above-men- 
tioned sections are provided separately and intercon- 
nected via communication facilities such as a MIDI 
interface, various networks and the like. Further, the 45 
waveform producing apparatus of the present invention 
may comprise a combination of a personal computer 
and application software, in which case various 
processing programs may be supplied to the waveform 
producing apparatus from a storage media such as a so 
magnetic disk, optical disk or semiconductor memory or 
via a communication network. Furthermore, the wave- 
form producing apparatus of the present invention may 
be applied to automatic performance apparatus such as 
a player piano. 55 
[0082] In summary, the present invention having 
been described so far is characterized in that a packet 
stream is generated by combining a plurality of packets 



corresponding to the received style-of-rendition identifi- 
cation information which are selected from among a 
number of packets usable for producing waveforms cor- 
responding to various styles of rendition and then a 
waveform having characteristics of the style of rendition 
indicated by the style-of-rendition identification informa- 
tion is produced on the basis of the generated packet 
stream. Thus, the present invention can produce a 
waveform corresponding to a desired style of rendition 
in a simplified manner with great facility. Further, 
because the time information of the packets is adjusted 
and a waveform can be produced on the basis of the 
adjusted time information, the present invention allows 
style-of-rendition waveforms rich in variations to be pro- 
duced with a simplified structure and increased control- 
lability. As a result, the present invention can 
advantageously produce high-quality waveforms, taking 
styles of rendition or articulation into account, in a sim- 
plified manner with significantly increased controllability. 

Claims 

1. A waveform producing method comprising the 
steps of: 

receiving style-of-rendition identification infor- 
mation representing a style of rendition of a 
performance tone; 

generating a packet stream in accordance with 
the style-of-rendition identification information 
received by said step of receiving, the packet 
stream including packets of information for pro- 
ducing and/or controlling a waveform; and 
producing a waveform on the basis of the 
packet stream generated by said step of gener- 
ating. 

2. A waveform producing method as claimed in claim 
1 wherein said packet stream includes a plurality of 
packets and time information of individual ones of 
the packets, and said step of producing a waveform 
includes a step of arranging the individual packets 
on a time axis on the basis of the time information. 

3. A waveform producing method as claimed in claim 
1 wherein said packet stream includes a plurality of 
packets and time information of individual ones of 
the packets, and said step of producing a waveform 
includes a step of adjusting the time information 
and a step of arranging the packets on a time axis 
on the basis of the time information adjusted by 
said step of adjusting. 

4. A waveform producing method as claimed in claim 
3 wherein said step of adjusting the time informa- 
tion adjusts the time information forward or back- 
ward by a predetermined amount 
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5. A waveform producing method as claimed in claim 
3 wherein said step of adjusting the time informa- 
tion adjusts the time information forward or back- 
ward by a predetermined amount on the basis of a 
random number. 5 

6. A waveform producing method as claimed in claim 
1 wherein said packet stream comprises data for 
controlling a pitch of a waveform to be produced. 

10 

7. A waveform producing method as claimed in claim 
1 wherein said packet stream comprises data for 
controlling an amplitude of a waveform to be pro- 
duced. 

15 

8. A waveform producing method as claimed in claim 
1 wherein said packet stream comprises data for 
controlling a shape of a waveform to be produced. 

9. A waveform producing apparatus comprising: 20 

means for receiving style-of-rendition identifi- 
cation information representing a style of rendi- 
tion of a performance tone; 
means for generating a packet stream in 25 
accordance with the style-of-rendition identifi- 
cation information received by said means for 
receiving, the packet stream including packets 
of information for producing and/or controlling a 
waveform; and 30 
means for producing a waveform on the basis 
of the packet stream generated by said means 
for generating. 

10. A waveform producing apparatus comprising: 35 

an input device adapted to input, into said 
waveform producing apparatus, style-of-rendi- 
tion identification information representing a 
style of rendition of a performance tone; and 40 
a processor coupled with said input device and 
adapted to: 

generate a packet stream in accordance with 
the style-of-rendition identification information 
inputted via said input device, the packet 45 
stream including packets of information for pro- 
ducing and/or controlling a waveform; and 
produce a waveform on the basis of the gener- 
ated packet stream. 

50 

11. A machine-readable storage medium containing a 
group of instructions to cause said machine to 
implement a waveform producing method, said 
waveform producing method comprising the steps 

of: 55 

receiving style-of-rendition identification infor- 
mation representing a style of rendition of a 



performance tone; 

generating a packet stream in accordance with 
the style-of-rendition identification information 
received by said step of receiving, the packet 
stream including packets of information for pro- 
ducing and/or controlling a waveform; and 
producing a waveform on the basis of the 
packet stream generated by said step of gener- 
ating. 

12. A waveform producing method comprising the 
steps of: 

receiving a packet stream including a plurality 
of packets and time information of individual 
ones of the packets; 

arranging, on a time axis, vector data for pro- 
ducing a waveform representing a style of ren- 
dition of a performance tone corresponding to 
the packets, in accordance with the time infor- 
mation; and 

producing a waveform on the basis of the vec- 
tor data arranged on the time axis. 

13. A waveform producing apparatus comprising: 

means for supplying a packet stream including 
a plurality of packets and time information of 
individual ones of the packets; 
means for arranging, on a time axis, vector 
data for producing a waveform representing a 
style of rendition of a performance tone corre- 
sponding to the packets/in accordance with the 
time information; and 

means for producing a waveform on the basis 
of the vector data arranged on the time axis. 

14. A waveform producing apparatus comprising: 

a processor adapted to: 

receive a packet stream including a plural- 
ity of packets and time information of indi- 
vidual ones of the packets; 
arrange, on a time axis, vector data for pro- 
ducing a waveform representing a style of 
rendition of a performance tone corre- 
sponding to the packets, in accordance 
with the time information; and 
produce a waveform on the basis of the 
vector data arranged on the time axis. 

15. A machine-readable storage medium containing a 
group of instructions to cause said machine to 
implement a waveform producing method, said 
waveform producing method comprising the steps 
of: 
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receiving a packet stream including a plurality 
of packets and time information of Individual 
ones of the packets; 

arranging, on a time axis, vector data for pro- 
ducing a waveform representing a style of ren- s 
dition of a performance tone corresponding to 
the packets, in accordance with the time infor- 
mation; and 

producing a waveform on the basis of the vec- 
tor data arranged on the time axis. 10 
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